Imagers such as still cameras and video cameras have a finite ability to resolve small differences in intensity, and are also only able to measure intensity accurately within a finite range. The range from the smallest detectable intensity above the noise to the largest intensity below saturation—commonly referred to as “dynamic range”—is therefore limited in conventional imagers. As a result, if a scene being imaged contains both very bright regions and very dark regions, it can be difficult to image the entire scene precisely and accurately.
One approach to solving this problem is to sequentially capture multiple images of the same scene using different exposures. The exposure for each image is typically controlled by varying either the F-number of the imaging optics or the exposure time of the image sensor. An image captured using a high exposure setting will tend to represent dark regions accurately but may exhibit saturation in bright regions. In contrast, an image captured using a low exposure setting will avoid saturation in bright regions but may be undesirably dark and noisy in the dark areas. Such differently exposed images can be combined into a single high dynamic range image—e.g., by using the high-exposure data for darker regions and the low-exposure data for brighter regions. The “dynamic range” of an image, as used herein, refers to the range from the lowest pixel brightness value in the image—corresponding to the smallest detected intensity—to the highest pixel brightness value in the image—corresponding to the largest detected intensity.
The above-described method is best suited to scenes that are static or very slowly changing. In particular, the locations and radiances of the scene objects must remain relatively constant during the capture of a sequence of images under different exposures. The imager itself must also be stationary. Only if the images are captured in rapid succession can they be fused accurately to derive a high dynamic range image or video sequence without substantial errors and artifacts.
The requirement of a static or slowly changing scene can be avoided by using multiple imaging systems. For example, in one system, a beam splitter arrangement is used to permit multiple imagers to simultaneously sense the incoming light field. Each imager has a preset exposure setting which is different from those of the other imagers. The exposure setting of each imager is typically set by using an optical attenuator or by adjusting the exposure time of the sensor. This approach has the benefit of producing accurate, high dynamic range images in real time, even if the scene objects and/or the imager move during image capture. However, such a system tends to be expensive because it requires multiple imagers, precision optics for the alignment of the acquired images, and additional hardware for the capture and processing of the multiple images.
An additional approach to high dynamic range imaging is based on a modification of the conventional charge coupled device (CCD) array. In the modified CCD array, each detector cell includes two sensing elements (potential wells) having different sizes and, accordingly, different saturation levels. When the modified CCD array is exposed to incoming light, two measurements of the light intensity are made within each cell. The measurements within each cell are combined on-chip before the image is read out. However, this technique is expensive because it entails fabrication of a sophisticated and costly CCD array. In addition, the spatial resolution of the CCD array is reduced, because each pair of light sensing elements occupies more chip area than a single light sensing element of a conventional array. In addition, such a system requires extra on-chip circuitry for combining the outputs (photogenerated charge) of the two light sensing elements of each cell.
An additional image sensing array which has been used for high dynamic range imaging is a solid state array of light sensing elements in which the light sensing element in each pixel position of the array includes a computational element for measuring the time required for a potential well associated with the light sensing element to reach its full charge capacity. The array is typically designed so that the photogenerated charge storage capacity is the same for light sensing elements at all pixel positions. Accordingly, the length of time in which any given potential well becomes filled with photogenerated charge is proportional to the irradiance at the pixel position of the associated light sensing element. The recorded time values are read out and converted to a high dynamic range image. At least one device having 32×32 light sensing elements has been implemented. This approach can be useful for low-resolution image sensors, but is difficult to scale up to high resolution image sensing arrays without greatly increasing fabrication costs. In addition, because exposure times tend to be long in dark scene regions, this technique tends to be susceptible to motion blur.
Yet another system uses an image sensing array with spatially varying pixel sensitivities to achieve high dynamic range. In this approach, respective groups of light sensing elements at neighboring pixel positions in the image sensing array are fabricated to have different, fixed sensitivities. Such an arrangement allows simultaneous sampling of the spatial dimensions of the scene and of the exposure dimension of image irradiance. In most images, because a variety of different pixel sensitivities are available in any given region of the array, even if the light sensing element at a particular pixel position is saturated, it is likely that at least one light sensing element at a neighboring pixel position is not saturated. Similarly, if the light sensing element at a particular pixel position is underexposed or even has a measured brightness of zero, it is likely that at least one light sensing element at a neighboring pixel position measures a non-zero brightness. However, a drawback of such a system is that some spatial resolution of the image is sacrificed to attain the increased brightness resolution. Moreover, because the exposure value at each pixel position is fixed, not every pixel is measured with full precision.